3-D Optical Microscope

ABSTRACT

A 3-D optical microscope, a method of turning a conventional optical microscope into a 3-D optical microscope, and a method of creating a 3-D image on an optical microscope are described. The 3-D optical microscope includes a processor, at least one objective lens, an optical sensor capable of acquiring an image of a sample, a mechanism for adjusting focus position of the sample relative to the objective lens, and a mechanism for illuminating the sample and for projecting a pattern onto and removing the pattern from the focal plane of the objective lens. The 3-D image creation method includes taking two sets of images, one with and another without the presence of the projected pattern, and using a software algorithm to analyze the two image sets to generating a 3-D image of the sample. The 3-D image creation method enables reliable and accurate 3-D imaging on almost any sample regardless of its image contrast.

RELATED APPLICATIONS

This application is a divisional of U.S. patent application Ser. No.11/754,282, entitled “3-D Optical Microscope” filed May 26, 2007.

FIELD OF THE INVENTION

This invention relates generally to an optical microscope and, moreparticularly, to a three-dimensional (3-D) optical microscope, and amethod of turning a conventional optical microscope into a 3-D opticalmicroscope.

BACKGROUND OF THE INVENTION

A conventional microscope enables an operator to view magnified imagesof minute features on a sample otherwise invisible to the human eye.Because of this, conventional microscopes have been widely used inuniversities, in research institutes, and in many industries. Aconventional microscope, however, has important limitations. Forexample, it only provides a two-dimensional (2-D) image of a samplewhile in the real world a majority of samples are 3-D in nature.

Various improvements have been made over the years to achieve 3-Dviewing and 3-D imaging with optical microscopes. Costales in U.S. Pat.No. 6,275,335 discloses a stereomicroscope using various polarizingoptical components to achieve a stereoscopic effect in the image.Although Costales' microscope produces a perception of depth, it cannotprovide quantitative measurement of the depth dimension.

Kino in U.S. Pat. No. 5,022,743 proposes a confocal microscope utilizinga spinning Nipkow disc. Sheppard in U.S. Pat. No. 4,198,571 discloses aconfocal microscope based on laser scanning. Although a confocalmicroscope is able to generate a 3-D image and provide quantitativedepth measurement, it is expensive to build and relatively complex tomaintain and operate. In addition, if one already bought a conventionalmicroscope, it is not easy and in many cases impossible to turned hismicroscope into a confocal microscope.

Sieckmann in U.S. Appl. No. 2004/0257360A1 proposes a method of creating3-D images of a sample by analyzing a stack of images of the sampletaken at various focus positions. Although it is cost effective toimplement such a method, it only works on samples with contrastingfeatures. In short, Sieckmann's method fails to generate a reliable andaccurate 3-D image of a sample with little or no contrast.

Morgan in U.S. Pat. No. 5,381,236 proposes an optical range sensor thatis capable of sensing the depth profile of a plain surface by activelyprojecting a pattern of light onto the target object. Although Morgan'ssensor measures the 3-D profile of a sample, it does not combine the 3-Dprofile with the intensity or color information of the sample. As aresult, his sensor does not yield a 3-D image. In addition, the patternof light in Morgan's sensor is always superimposed on the samplesurface, and thus interferes with the true features of the samplesurface being captured by a camera.

Accordingly, there is a need for a 3-D optical microscope that isrelatively low cost to build and easy to operate; a method that can beeasily deployed to turn a conventional microscope into a 3-D opticalmicroscope; and a 3-D imaging method that works on all samplesregardless of their feature contrast.

SUMMARY OF THE INVENTION

The need is met with the present invention which achieves threeobjectives: first, to create a simple and relatively low cost microscopethat is capable of generating a 3-D image on samples with or withoutcontrast; second, to propose simple hardware modifications that one canmake to turn a conventional optical microscope into a 3-D opticalmicroscope; and third, to disclose a method that enables reliable andaccurate 3-D imaging on almost any sample regardless of its imagecontrast.

In a first aspect of the present invention, a 3-D optical microscopeincludes at least one objective lens to form an optical image of asample; an optical sensor capable of acquiring said optical image; meansfor adjusting focus position of said sample relative to said objectivelens; means for illuminating said sample, and for projecting a patternonto and removing said pattern from the focal plane of said objectivelens; and a processor connected to said microscope to control saidoptical sensor, said means for adjusting focus position, and said meansfor projecting and removing said pattern.

In a second aspect of the present invention, a conventional microscopeis used as a base unit. Several modifications are performed on the baseunit to turn it into a 3-D optical microscope. Said modificationscomprise of: (1) providing a means to projecting a pattern onto and toremoving said pattern from the focal plane of an objective lens of saidmicroscope. (2) Providing a means to automating objective lens focusadjustment. (3) Providing an optical sensor capable of acquiring animage if said optical sensor is not already a part of said conventionalmicroscope. (4) Providing a processor connected to said microscope tocontrol said optical sensor, said means to automating objective lensfocus adjustment, and said means to projecting a pattern onto and toremoving said pattern from the focal plane of said objective lens.

In a third aspect of the present invention, a method of creating a 3-Dimage with a 3-D optical microscope of the present invention isdisclosed. The method comprises: (a) projecting a pattern onto the focalplane of an objective lens of said optical microscope. (b) Varying therelative distance between a sample and said objective lens atpre-determined steps, and capturing a first set of images of said sampleat each step, and recording said relative distance at each step. (c)Removing said pattern from the focal plane of said objective lens,resetting image sensor parameters, retracing the same steps taken in(b), and capturing a second set of images of said sample at each step.(d) Using a software algorithm to analyze the first and the second setof images to create a 3-D image of said sample.

Finally, in a fourth aspect of the present invention, a method ofcreating a 3-D image with a 3-D optical microscope of the presentinvention is disclosed. The method comprises: (1) varying the relativedistance between a sample and an objective lens of said microscope atpre-determined steps. (2) At each step, projecting a pattern created byan electronic pattern generator onto the focal plane of said objectivelens and capturing a first image of said sample; removing said patternfrom the focal plane of said objective lens and capturing a second imageof said sample; storing said first image into a first image array andsaid second image into a second image array. (3) Using a softwarealgorithm to analyze the first and the second image array to create a3-D image of said sample.

Other aspects and advantages of the present invention will becomeapparent from the following detailed description, taken in conjunctionwith the accompanying drawings, illustrating by way of example theprinciples of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a block diagram illustrating a 3-D optical microscope with areflective illuminator in accordance with a first embodiment of thepresent invention.

FIG. 1B is a diagram illustrating a mechanical pattern generator.

FIG. 2 is a flowchart illustrating a process of extracting a 2-Dcontrast array from a 2-D image using maximum gradient method.

FIG. 3 is a flowchart illustrating a process of performing a Z scan andimage capture.

FIG. 4 is a flowchart illustrating a process of generating a 3-D imagebased on image contrast.

FIG. 5A is a flowchart illustrating a two-pass Z scan and imageacquisition process associated with a mechanical pattern generator ofthe present invention.

FIG. 5B is a flowchart illustrating a single pass Z scan and imageacquisition process associated with an electronic pattern generator ofthe present invention.

FIG. 5C is a flowchart illustrating a data analysis process to generatea 3-D image in accordance with the present invention.

FIG. 6 is block diagram illustrating a 3-D optical microscope with areflective illuminator in accordance with a second embodiment of thepresent invention.

FIG. 7 is a block diagram illustrating a 3-D optical microscope with atransmitted illuminator in accordance with a third embodiment of thepresent invention.

FIG. 8 is a block diagram illustrating a 3-D optical microscope with atransmitted illuminator in accordance with a fourth embodiment of thepresent invention.

FIG. 9A is a diagram illustrating major components of a conventionaloptical microscope with a reflective illuminator.

FIG. 9B is a diagram illustrating modifications made to the conventionaloptical microscope of FIG. 9A to turn it into a 3-D optical microscopein accordance with a fifth embodiment of the present invention.

FIG. 9C is a diagram illustrating modifications made to the conventionaloptical microscope of FIG. 9A to turn it into a 3-D optical microscopein accordance with a sixth embodiment of the present invention.

FIG. 10A is a diagram illustrating major components of a conventionaloptical microscope with a transmitted illuminator.

FIG. 10B is a diagram illustrating modifications made to theconventional optical microscope of FIG. 10A to turn it into a 3-Doptical microscope in accordance with a seventh embodiment of thepresent invention.

FIG. 10C is a diagram illustrating modifications made to theconventional optical microscope of FIG. 10A to turn it into a 3-Doptical microscope in accordance with a eighth embodiment of the presentinvention.

FIG. 10D is a diagram illustrating modifications made to theconventional optical microscope of FIG. 10A to turn it into a 3-Doptical microscope in accordance with a ninth embodiment of the presentinvention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1A is a diagram illustrating a 3-D optical microscope with areflective illuminator in accordance with a first embodiment of thepresent invention. A microscope operating in reflective illuminationmode is often used for studying opaque samples such as a semiconductorwafer. Reflective illuminator 100 provides illumination for themicroscope and contains several components. Light source 101 generatesilluminating light. Among possible choices for the light source are: alamp, a fiber coupled light, a LED light, a laser, and etc. In thepreferred embodiment, either a halogen lamp or a fiber coupled lightsource is used. Lenses 103, 105, and 107 ensure uniform illumination onsample 120 and, when pattern generator 115 is placed in the illuminatoroptical path, project an image of a patterned article to the focal planeof objective lens 110. Beam-splitter 109 is mounted at a 45° angle withrespect to a centerline connecting lenses 103, 105, and 107 so thatlight from the light source is directed vertically downward to objectivelens 110. Positioning means 130 is provided to change the relativeposition between sample 120 and objective lens 110. As a result,different features on the sample can be brought into focus of objectivelens 110. As an option, a XY stage (not shown) can be incorporated intothe microscope of FIG. 1A to move sample 120 around in a horizontalplane. In the preferred embodiment, positioning means 130 is a motorizedZ stage. There are, of course, other ways to vary the relative positionbetween sample 120 and objective lens 110. For example, objective lens110 could be mounted on a piezoelectric actuator. In such anarrangement, sample 120 remains stationary while objective lens 110moves up and down. It is understood by those skilled in the art thatthese variations are within the scope of this invention. Coupler 140 inconjunction with objective lens 110 yields an image of sample 120 onoptical sensor 150. In the preferred embodiment, optical sensor 150 iseither a CCD or a CMOS camera.

Pattern generator 115 can either be a mechanical one or an electronicone. An example of an electronic pattern generator is a liquid crystalpattern generator. One possible implementation of a mechanical patterngenerator is shown in FIG. 1B. It contains several components. Patternedarticle 104 has a pre-determined pattern shown in FIG. 1B as a grid.Other patterns will also work as long as they satisfy the followingconditions: (1) they have high contrast; (2) they are either regular orrandom but evenly distributed; (3) they are semi-transparent; (4) theirminimum feature size matches sampling resolution of an imaging opticalsensor used. Patterned article 104 could be a piece of patternedphotographic film or patterned glass. It is understood that variouspatterns can be created on various substrate materials to make patternedarticle 104, and that these variations are also within the scope of thisinvention.

Patterned article 104 is seated in hardware mount 102, so is a regularmicroscope field-stop diaphragm 106. The field-stop diaphragm can eitheropen up or close down to control the diameter of the illuminating lightbeam. When inserted into the illuminator optical path, the field-stopdiaphragm can improve an image by reducing interference caused by out offield-of-view objects. Patterned article 104 and field-stop diaphragm106 share the same hardware mount 102 for a reason. We mentioned earlierthat an image of patterned article 104, when inserted in the opticalpath of illuminator 100, is formed at the focal plane of objective lens110. A field-stop by definition must be placed at a location within amicroscope illuminator where an image of the field-stop is formed at thefocal plane of an objective lens. This unique location within amicroscope illuminator is called the field-conjugate plane. By attachingpatterned article 104 and field-stop diaphragm 106 on the same hardwaremount 102, and by sliding mount 102 perpendicular to the optical path ofilluminator 100, we make sure that both the patterned article and thefield-stop diaphragm stays at the field-conjugate plane when they areinserted in the illuminator optical path. Mount 102 is controlled bypositioning means 108 so that patterned article 104 and field-stop 106can be inserted into and taken out of the illuminator optical path whenneeded. In the preferred embodiment, positioning means 108 is amotorized actuator. Other positioning means such as a solenoid actuatorwill work as well. By mechanically sliding hardware mount 102, we caneither insert patterned article 104 into the illuminator optical path,thus projecting an image of patterned article 104 onto the focal planeof the objective lens; or we can remove patterned article 104 from theilluminator optical path, thus removing the image of patterned article104 from the focal plane of the objective lens. Since insertion andremoval of the image of the patterned article is done mechanically, wecall such a device a mechanical pattern generator.

If pattern generator 115 is an electronic one such as a liquid crystalpattern generator (or LCPG), one can just place the LCPG in theilluminator optical path. The LCPG is capable of creating a desiredpattern similar to that of patterned article 104. In addition, the LCPGcan generate images that mimic the opening up and closing down of thediaphragm of field-stop 106. In short, the LCPG can replace thefunctionality of the entire assembly of FIG. 1B. Since creation andremoval of a pattern on the LCPG is done electronically, there is noneed to physically slide the LCPG in and out. Thus, we call a devicelike the LCPG an electronic pattern generator.

There are alternative ways to design reflective illuminator 100. Forexample, one can use a light source and less than three lenses toproject an image of a patterned article onto the focal plane of anobjective lens. One can also use a light source and more than threelenses to create multiple field-conjugate planes where he can place apatterned article and have its image projected onto the focal plane ofan objective lens. It is understood that these alternative illuminatordesigns are also within the scope of the present invention.

Processor 160 is connected to the 3-D optical microscope of FIG. 1A.Said processor is used to control positioning means 130, patterngenerator 115, and optical sensor 150. In addition, said processoranalyzes data and creates a 3-D image of the sample. In the preferredembodiment, said process is a personal computer.

Creating a 3-D image on the 3-D optical microscope of FIG. 1A with amechanical pattern generator shown in FIG. 1B involves two separatepasses according to the present invention. In the first pass, patternedarticle 104 is inserted into the optical path of illuminator 100.Positioning means 130 moves sample 120 from a pre-determined startposition away from objective lens 110 through a set of pre-determinedsteps. At each step, optical sensor 150 captures and saves the image ofthe sample. In the second pass, patterned article 104 is removed fromthe optical path of illuminator 100. Positioning means 130 moves sample120 to the same start position and subsequently through the same stepsas defined in the first pass. At each step, optical sensor 150 capturesand saves the image of the sample. When done, processor 160 analyzes thefirst and second pass data set to create a 3-D image. The details of thetwo-pass 3-D image creation process of the present invention will bediscussed in depth when we describe FIGS. 5A and 5C later.

Creating a 3-D image on the 3-D optical microscope of FIG. 1A with anelectronic pattern generator such as a LCPG involves just a single passaccording to the present invention. During the process, positioningmeans 130 moves sample 120 from a pre-determined start position awayfrom objective lens 110 through a set of pre-determined steps. At eachstep, a pattern created by the LCPG is projected onto the focal plane ofobjective lens 110, optical sensor 150 captures and saves a first imageof the sample; then the LCPG quickly erases that pattern, optical sensor150 captures and saves a second image of the sample. This processrepeats itself until all the steps have been taken. When done, processor160 analyzes the first and second image set to create a 3-D image. Thedetails of the single pass 3-D image creation process of the presentinvention will be discussed when we describe FIGS. 5B and 5C later. Itis understood that the single pass 3-D image creation process can alsobe applied to the 3-D optical microscope of FIG. 1A with a mechanicalpattern generator as long as insertion and removal of the patternedarticle is sufficiently fast. In the next several paragraphs, we willdiscuss software controls and algorithms related to acquiring 2-D imagestacks of a sample, extracting image contrast, constructing 3-D depthprofiles, and creating a 3-D rendering of the sample.

A microscope objective lens is usually characterized by severalimportant parameters such as focal length, magnification, workingdistance (W.D.), and numerical aperture (N.A.). To a large extent, theN.A. of an objective lens determines its depth-of-field. When a sampleis placed at the focal plane of an objective lens, the sample is said tobe in-focus, and the image produced by the objective lens has thesharpest contrast. If the same sample is placed slightly away from thefocal plane but is still within the range defined by the depth-of-field,the image contrast is still relatively sharp. As the sample is moved outof the depth-of-field range, it becomes out-of-focus, and the imagebecomes blurrier.

Mathematically, image contrast is related to the high frequency orgradient content of the image intensity: the sharper the image, thehigher the intensity gradient, and the stronger its high frequencycontent. Consider a microscope operator who is trying to find the bestfocus. He will move the sample up and down around the focal plane of theobjective lens to find the point where the image contrast is thehighest. Similarly, a system can be devised so that the relativeposition between the sample and the objective lens is changed atcontrolled steps. After each step move, a camera takes an image; theimage is converted into digital form so a computer can extract its highfrequency content. This high frequency content is recorded and comparedwith that of the previous steps. As the sample is stepping one-waytowards and eventually passing through the best focus, its image's highfrequency content level would rise, reach a peak, and then fall. Thebest focus position corresponds to where the image's high frequencycontent reaches a maximum.

Generally, an object is not flat but rather has a depth profile. Bycalculating the high frequency contents at each pixel for every imagetaken at a specific distance between the sample and the objective lens,the computer can compare and find the distance where maximum highfrequency content of each pixel occurs. By applying this calculation toall pixels, the process can, in theory, yield a depth profile of thesample. The intensity or color values of those pixels that are locatedon the contour of the depth profile can also be extracted from therelevant images. Graphic rendering of both depth and color informationshould yield a 3-D image of the sample. This type of image contrastbased 3-D creation method forms the bases of Sieckmann in U.S. Appl. No.2004/0257360A1.

There are many well-known methods in calculating the high frequencycontent of a pixel. Most of these methods are based on finding theintensity differences among neighboring pixels, and are called high passfilters, or gradient filters. Most commonly, the operation of thesefilters is a convolution of a filter mask with the pixel and itsimmediate 8 neighboring pixels:

$\begin{matrix}{{{Filter}\mspace{14mu} {mask}},{Laplacian}} & {{Neighboring}\mspace{14mu} {pixels}} \\\begin{matrix}{- 1} & {- 1} & {- 1} \\{- 1} & 8 & {- 1} \\{- 1} & {- 1} & {- 1}\end{matrix} & \begin{matrix}{P( {{- 1},{- 1}} )} & {P( {0,{- 1}} )} & {P( {1,{- 1}} )} \\{P( {{- 1},0} )} & {P( {0,0} )} & {P( {1,0} )} \\{P( {{- 1},1} )} & {P( {0,1} )} & {P( {1,1} )}\end{matrix}\end{matrix}$

Where P(i, j) is the intensity of a pixel located (i, j) pixels awayfrom the reference pixel or pixel of interest, and i being the relativepixel number in the horizontal (X) direction, and j being that in thevertical (Y) direction. For example, if P(0, 0) is the intensity of thepixel of interest, then P(−1,−1) refers to that of its top leftneighboring pixel, P(0,−1) that of its top neighbor, and P(1,−1) that ofits top right neighbor. Using the Laplacian high pass filter to find thehigh frequency content of P(0, 0) involves convolving the Laplacianfilter mask with the neighboring pixels:

High frequency content of P(0, 0)=absolute (8*P(0,0)−

1*P(−1,−1)−1*P(0,−1)−1*P(1,−1)−

1*P(−1,0)−1*P(1,0)−

1*P(−1,1)−1*P(0,1)−1*P(1,1))  Equation 1

A high pass filter, like the Laplacian, is non-directional in that itdoes not care whether or not an image has directional features. Thereare directional high pass or edge filters that have filter masksoptimized to the direction of interest. For example, if an image isknown to have horizontal edges or lines, and if only the verticalcomponent of the high frequency content is wanted, then the North-Southedge filter mask can be used to produce a much stronger result than onecan get by using a non-directional filter mask.

North-South  directional  filter  mask: $\begin{matrix}{- 1} & {- 1} & {- 1} \\{- 0} & 0 & 0 \\1 & 1 & 1\end{matrix}$

Since most images do not have a fixed feature orientation, a singledirectional filter operation will in general not yield desirableresults. Instead, multiply applications of directional filters such asthe ones shown below, each with a different direction, are performed tofind the gradients along these directions. Among them, the one filterthat yields maximum value determines the maximum high frequency contentof a pixel. Such an approach is called maximum gradient method.

$\begin{matrix}\begin{matrix}{{North}\text{-}{South}\mspace{14mu} {mask}\text{:}} \\\begin{matrix}{- 1} & {- 1} & {- 1} \\0 & 0 & 0 \\1 & 1 & 1\end{matrix}\end{matrix} & \begin{matrix}{{East}\text{-}{West}\mspace{14mu} {mask}\text{:}} \\\begin{matrix}1 & 0 & {- 1} \\1 & 0 & {- 1} \\1 & 0 & {- 1}\end{matrix}\end{matrix} & \begin{matrix}{135\mspace{14mu} \deg \mspace{14mu} {mask}\text{:}} \\\begin{matrix}{- 1} & {- 1} & 0 \\{- 1} & 0 & 1 \\0 & 1 & 1\end{matrix}\end{matrix} & \begin{matrix}{45\mspace{14mu} \deg \mspace{14mu} {mask}\text{:}} \\\begin{matrix}0 & {- 1} & {- 1} \\1 & 0 & {- 1} \\1 & 1 & 0\end{matrix}\end{matrix}\end{matrix}$

A three-by-three filter mask is commonly used because of its small size,thus computational efficient, and because it captures only the highestfrequency content. Other directional 3×3 edge filter masks are alsopossible. For example, the Sobel edge filter mask, shown below, willyield similar results.

$\begin{matrix}{{Sobel}\mspace{14mu} {North}\text{-}{South}\mspace{14mu} {mask}\text{:}} \\\begin{matrix}1 & 2 & 1 \\0 & 0 & 0 \\{- 1} & {- 2} & {- 1}\end{matrix}\end{matrix}$

A larger filter mask, 5×5 or 7×7 for example, also works. In particular,a larger filter mask can be tailored to work better for images withlower contrast, or with more lower frequency contents, at the expense ofcomputation efficiency. It is understood that other filter masks thatcan be used to extract contrast information is within the scope of thepresent invention.

A process of extracting a 2-D contrast array from a 2-D image based onmaximum gradient method is illustrated in FIG. 2. In step 201, a default2-D ContrastArray(x, y) is created, where all pixel are either leftun-initialized or set to a default value. In steps 202 and 203, theinitial pixel location is set to (x=1, y=1). In step 204, a test arrayis created by copying the intensity values of a 3-by-3 section of theimage pixels centering around pixel (1, 1). Convolving this test arraywith a North-South mask in a manner similar to Equation 1 yields a valueC1 in step 205. Subsequently, convolving the same test array with anEast-West, 135-deg, and 45-deg masks yield values C2, C3, and C4respectively in steps 206-208. In step 209, a comparison is made amongC1, C2, C3, and C4 to find the maximum absolute value. In step 210, themaximum contrast for pixel (1, 1), ContrastArray(1, 1), is set to equalto the maximum value found in step 209. Next, we move one pixel over inthe horizontal direction, that is, our next pixel is (2, 1). Since x=2,x is less than (image width −1), test 211 returns YES and the processflow is looped back to 204. This loop will run its course until x (imagewidth −1). At that point, the flow proceeds to step 212 where we startto increment y pixel number from 1 to 2, and the process is moved backto step 203, and x is set to 1. Again the loop will run its course untily (image height −1) and test 212 returns NO. At that moment we havefilled in the entire ContrastArray(x, y) except for the border pixels.We simply fill ContrastArray corresponding to the border pixels with thevalues of their adjacent pixels in step 213. We now have finishedgenerating a 2-D contrast array from a 2-D image.

To create a 3-D depth profile of a sample based on contrast analysismethod, a stack of images must be taken at different focus positions orrelative sample-objective lens z distances. The detailed image capturingprocess is illustrated in FIG. 3. The process starts with step 301, thecreation of a default IntensityArray(x, y, z) where all pixels areeither left un-initialized or set to a default value. At this point, auser must determine the range of z distances in order to cover thetallest peak and the deepest valley on the sample. This is done in steps302 and 303. In step 304, the user also needs to specify the number ofsteps, so the computer can calculate the Z step size needed to cover theentire z distance range. Alternatively, he can specify step size, andthe computer determines the number of steps to cover the z range. Theuser must also choose the right amount of illumination and selectappropriate gain and exposure time for the camera so that an imagecaptured by the camera is neither too bright nor too dark. Of course,once all of these settings are fine tuned, they can be stored in arecipe to be used later by the computer on similar samples. Afterchoosing all the settings, the sample is moved to the starting Z scanposition in step 305. In step 306, the camera captures an image indigital form. The intensity values of every pixel of the image and thecorresponding z distance value is stored into IntensityArray(x, y, z).At that point, the flow proceeds to check point 307 to see if the last Zscan step is reached. If not, the sample is moved down one step, at 308,and the process is directed back to step 306. This loop will run itscourse until the last Z scan step is reached. The process of FIG. 3 iscalled Z scan and image capture.

Since the working distance of a high magnification objective lens isusually very small, some precaution is needed to prevent the objectivelens from coming into contact with the sample during the Z scan.Generally speaking, it is safer to start the Z scan from a positionwhere the sample is closest to the objective lens within the Z scanrange, and gradually move the sample away from the objective lens. Inthe case of FIG. 3, this means to start the scan from the bottom of thesample, and move the sample down to cover the full Z scan range. Asmentioned earlier, there are alternative ways to carry out the Z scanother than moving the sample. For example, it is possible to move theobjective lens up and down to achieve the same result as stepping thesample. If the approach is moving the objective lens, it is safer tostart the Z scan from the bottom of the sample, and then gradually stepthe objective lens upward to cover the full Z scan range.

Once a stack of images is captured through the Z scan process of FIG. 3,a 3-D image of the sample can be constructed using a process outlined inFIG. 4. The process starts with step 401, the creation of a highcontrast array ContrastArray(x, y, z). For every z step, thecorresponding values of ContrastArray(x, y, z) is calculated based onthe maximum gradient method of FIG. 2 using the image data stored inIntensityArray(x, y, z) of FIG. 3. The next step, 402, is to create adefault array 3DZ(x, y) to store depth value and a default array 3DI(x,y) to store intensity value. In steps 403 and 404, the pixel location(x, y) is initially set at x=0 and y=0. For this initial pixel (0, 0), asearch is carried out to find the maximum value among array elementsContrastArray(0, 0, z). The z value corresponding to this maximum isdefined as Z_(max) and is then stored as array element 3DZ(0, 0) in step405. The intensity value corresponding to IntensityArray(0, 0, Z_(max))is stored as array element 3DI(0, 0) in step 406. It is now time to moveto the next pixel x=1 and y=0, or pixel (1, 0) in step 407. Test 408 iscarried out to see if x is less than the image width. A positive answerwill direct the flow back to steps 405 through 408. This loop will runits course until test 408 yields a negative answer. At that point 409,the y pixel number is incremented by 1. Test 410 is carried out to seeif y is less than the image height. A positive answer will direct theflow back to steps 404 through 410. This loop will run its course untiltest 410 yields a negative answer. At that moment, array 3DZ(x, y) andarray 3DI(x, y) are filled. The final step 411 involves taking the zvalue of 3DZ(x, y) and image intensity value 3DI(x, y) at every pixellocation (x, y) and rendering them as a 3-D image.

While contrast based 3-D image creation method, as described during thediscussion of FIGS. 2 through 4, works for samples with a surfacetexture that produces high image contrast when in focus, it hasdifficulties with smooth samples with little contrast. Such an importantlimitation is unfortunately associated with Sieckmann in U.S. Appl. No.200410257360A1. An improved 3-D image generation method of the presentinvention that overcomes this difficulty will now be described. Ourmethod involves a two-pass image acquisition process if a mechanicalpattern generator is used or a one-pass image acquisition process if anelectronic pattern generator is used, and subsequent data analysis. FIG.5A outlines a two-pass image acquisition process of the presentinvention with a mechanical pattern generator. FIG. 5B outlines aone-pass image acquisition process of the present invention with anelectronic pattern generator. FIG. 5C illustrates a data analysisprocess to construct a 3-D image in according with the presentinvention.

In a two-pass image acquisition process of the present inventionassociated with a mechanical pattern generator, the first pass begins atstep 501 of FIG. 5A. Patterned article 104 is inserted in the opticalpath of illuminator 100 at the field-conjugate plane location in step502. For a flat sample with no contrast such as a polished clean baresilicon wafer surface, it is normally a challenge to know when thesurface is in focus. With patterned article 104 inserted, however, thetask becomes trivial. Whenever the pattern of patterned article 104 isin focus, we know that the flat sample is also in focus, and vice versa.For a sample with topography or depth profile, whenever a certain partof it, say region A, is in focus, the corresponding part of the patternof patterned article 104 which overlaps with region A in the imagefield-of-view will also be in focus. In step 503, the Z scan process ofFIG. 3, namely steps 301 through 309, is carried out to complete theremaining tasks of the first image acquisition pass. At that point,IntensityArray(x, y, z) is filled with image data from the first pass.

During the second image acquisition pass which begins at step 504,patterned article 104 is removed from the optical path of illuminator100 in step 505. In step 506, gain and exposure time of the camera arereset. This is necessary because the brightness of an image is quitedifferent between with and without the presence of patterned article104. Because of such a difference, a means to changing the imageintensity for the second pass has to be implemented. There are severalways to compensate for the intensity difference:

-   1. Using a neutral density attenuator    -   Instead of inserting a patterned article for the first pass and        removing the patterned article for the second pass, a neutral        density filter is inserted during the second pass. The        requirement for the neutral density attenuator is such that when        inserted, the amount of average image intensity change matches        that caused by inserting a patterned article.-   2. Using different camera settings    -   This approach calls for storing the image intensity prior to the        insertion of a patterned article. During the second pass when        the patterned article is removed, the gain and exposure time of        the camera is reset so that the image intensity matches that of        the stored intensity.-   3. Adjusting light source intensity    -   This approach requires saving the image intensity prior to the        insertion of a patterned article. During the second pass when        the patterned article is removed, the light source intensity is        adjusted so that the image intensity matches that of the stored        intensity.

Since gain and exposure control are provided on most cameras, the secondapproach above is the preferred embodiment of the present invention tocompensate for intensity difference between with and without thepresence of patterned article 104. In step 507, a defaultIntensityArray2(x, y, z) is created. In step 508, the Z scan startingpoint, the number of Z steps, and the step size of the second pass areset identical to that of the first pass. In step 509, the sample ismoved to the starting Z scan position. In step 510, the camera capturesan image in digital form, and the intensity or color values of everypixel of the image and the corresponding z distance value is stored intoIntensityArray2(x, y, z). At that point, the flow proceeds to checkpoint 511 to see if the last Z scan step is reached. If not, the sampleis moved down one step, in 512, and the process is directed back to step510. This loop will run its course until the last Z scan step isreached. At that point, IntensityArray2(x, y, z) is filled with imagedata from the second pass.

In a one-pass image acquisition process of the present inventionassociated with an electronic pattern generator, the process begins atstep 520 of FIG. 5B with the creation of two default arraysIntensityArray(x, y, z) and IntensityArray2(x, y, z). In step 521,preparation for the Z scan is carried out by following the steps of 302through 305 in FIG. 3, and the sample is moved to the starting positionof the Z scan. If the sample has little or no contrast, the electronicpattern generator can temporarily create a pattern and have it projectedonto the focal plane of the objective lens to help the search for thetop and bottom of the Z scan range. In step 522, the electronic patterngenerator creates a pattern and an image of the pattern is projectedonto the focal plane of the objective lens. In step 523, a first imageof the sample at the current z position is captured and saved inIntensityArray(x,y,z). This image contains information from both thesample and the pattern generated by the electronic pattern generator. Instep 524, the electronic pattern generator erases the pattern. As aresult, the focal plane of the objective lens now only containsinformation of the sample. In the mean time, the electronic patterngenerator also adjusts the intensity scale automatically so that theaverage image intensity with or without the pattern's presence is thesame. In step 525, a second image of the sample is captured and saved inIntensityArray2(x, y, z). In step 526, a test is carried out to see ifthe last Z scan step is reached. If not, the sample is moved down onestep, in 527, and the process is directed back to step 522. This loopwill run its course until the last Z scan step is reached. At thatpoint, IntensityArray(x, y, z) is filled with image data from the firstimage set, and IntensityArray2(x, y, z) is filled with image data fromthe second image set.

FIG. 5C outlines a data analysis process involved in constructing a 3-Dimage according to the present invention. The process starts with 530,the creation of a high contrast array ContrastArray(x, y, z) fromIntensityArray(x, y, z). Note that IntensityArray(x, y, z) andIntensityArray2(x, y, z) are fundamentally different; while the formeris based on images containing information from both the sample and thepatterned article, the latter is based on images containing informationfrom the sample only. The next step, 531, is to generate a 3DZ(x, y)from ContrastArray(x, y, z) according the procedure of FIG. 4. In steps532 and 533, the pixel location (x, y) is initially set at y=0 and x=0.For this initial pixel (0, 0), the value of 2-D array element 3DI(0, 0)is set equal to IntensityArray2(0, 0, z) in step 534, where z is thevalue of element 3DZ(0, 0). It is now time to move to the next pixel x=1and y=0, or pixel (1, 0), in step 535. Test 536 is carried out to see ifx is less than the image width. A positive answer will direct the flowback to steps 534 through 536. This loop will run its course until test536 yields a negative answer. At that point 537, the y pixel number isincremented by 1. Test 538 is carried out to see if y is less than theimage height. A positive answer will direct the flow back to steps 533through 538. This loop will run its course until test 538 yields anegative answer. At that moment, array 3DI(x, y) are filled with imageinformation from IntensityArray2(x, y, z). The final step 539 involvestaking the z values of 3DZ(x, y) and image intensity or color values3DI(x, y) at every pixel location (x, y) and rendering them as a 3-Dimage.

Those skilled in the art of computer programming and image processingwill be familiar with techniques for improving the computationalefficiency of the algorithm disclosed above. In particular, the use ofparallel programming to speed up the process of image capturing,processing, and storage is within the scope of this invention.

It is worth pointing out that the z values of 3DZ(x, y) are basedentirely on IntensityArray(x, y, z) while the image intensity or colorvalues of 3DI(x, y) is generated with data only from IntensityArray2(x,y, z). In essence, for the two-pass image acquisition process associatedwith a mechanical pattern generator, we are using the first pass data tofind a 3-D depth profile or a 3-D skeleton of the sample, and thenfilled the skeleton with image intensity or color data from the secondpass; for the one-pass image acquisition process associated with anelectronic pattern generator, we are using the first image set to find a3-D skeleton of the sample, and then filled the skeleton with imageintensity or color data from the second image set. The most importantdifference between the 3-D creation method of the present invention andthat of Sieckmann in U.S. Appl. No. 2004/0257360A1 lies in the fact thatin generating a 3-D skeleton of a sample, we rely on the image contrastof a patterned article while Sieckmann relies on the image contrast ofthe sample itself. Therefore, the method of the present invention willwork on samples with little or no image contrast while that ofSieckmann's won't.

FIG. 6 is a diagram illustrating a 3-D optical microscope with areflective illuminator in accordance with a second embodiment of thepresent invention. The main difference between this embodiment and thefirst embodiment illustrated in FIG. 1A is in illuminator design.Illuminator 600 provides illumination for the microscope and containsseveral components, one of which is an electronic pattern generator 604operating in reflective mode. One example of a reflective electronicpattern generator is a digital micro-mirror device (or DMD) made byTexas Instruments. A DMD contains a rectangular array of micro mirrors,each mounted on a tiny hinge that enables it to tilt either towards thelight source (ON) or away from it (OFF). When every micro mirror is ON,the DMD effectively becomes a regular mirror. When some micro mirrorsare ON while others are OFF, the DMD acts like a patterned article.Light source 601 generates illuminating light. Among possible choicesfor the light source are: a lamp, a fiber coupled light, a LED light, alaser, and etc. In the preferred embodiment, either a halogen lamp or afiber coupled light source is used. Lenses 602, 603, and 605 ensureuniform illumination on sample 120 and, when electronic patterngenerator 604 acts like a patterned article, project an image of thepatterned article to the focal plane of objective lens 110.Beam-splitter 606 is mounted at a 45° angle with respect to a horizontaldirection so that light from the light source is directed verticallydownward to objective lens 110.

Positioning means 130 is provided to change the relative positionbetween sample 120 and objective lens 110. As a result, differentfeatures on the sample can be brought into focus of objective lens 110.In the preferred embodiment, positioning means 130 is a motorized Zstage. As an option, a XY stage (not shown) can be incorporated into themicroscope of FIG. 6 to move sample 120 around in a horizontal plane.There are, of course, other ways to vary the relative position betweensample 120 and objective lens 110. For example, objective lens 110 couldbe mounted on a piezoelectric actuator. In such an arrangement, sample120 remains stationary while objective lens 110 moves up and down. It isunderstood by those skilled in the art that these variations are withinthe scope of this invention. Coupler 140 in conjunction with objectivelens 110 yields an image of sample 120 on optical sensor 150. In thepreferred embodiment, optical sensor 150 is either a CCD or a CMOScamera. Processor 160 is connected to the 3-D optical microscope of FIG.6. Said processor is used to control positioning means 130, electronicpattern generator 604, and optical sensor 150. In addition, saidprocessor analyzes data and creates a 3-D image of the sample. In thepreferred embodiment, said process is a personal computer.

A third embodiment of the present invention, shown in FIG. 7, is a 3-Doptical microscope with a transmitted illuminator. A microscopeoperating in transmitted illumination mode is often used for studyingtransparent objects such as biology related samples. Transmittedilluminator 700 provides illumination for the microscope and containsseveral components. Light source 701 generates illuminating light. Amongpossible choices for the light source are: a lamp, a fiber coupledlight, a LED light, a laser, and etc. In the preferred embodiment,either a halogen lamp or a fiber coupled light source is used. Lenses703, 705, 707, and 711 ensure uniform illumination on sample 720 and,when pattern generator 115 is placed in the illuminator optical path,project an image of the patterned article to the focal plane ofobjective lens 710. Beam-splitter 709 is mounted at a 45° angle withrespect to a centerline connecting lenses 703 and 705 so that light fromthe light source is directed vertically upward to the objective lens710.

Positioning means 730 is provided to change the relative positionbetween sample 720 and objective lens 710. As a result, differentfeatures on the sample can be brought into focus of objective lens 710.As an option, a XY stage (not shown) can be incorporated into themicroscope of FIG. 7 to move sample 720 around in a horizontal plane.Condenser lens 711 and sample 720 moves in tandem under the command ofpositioning means 730. In the preferred embodiment, positioning means730 is a motorized Z stage. There are, of course, other ways to vary therelative position between sample 720 and objective lens 710. Forexample, objective lens 710 could be mounted on a piezoelectricactuator. In such an arrangement, the sample remains stationary whilethe objective lens moves up and down. It is understood by those skilledin the art that these variations are within the scope of this invention.Coupler 740 in conjunction with objective lens 710 yields an image ofthe sample on optical sensor 750. In the preferred embodiment, opticalsensor 750 is either a CCD or a CMOS camera. Processor 160 is connectedto the 3-D optical microscope of FIG. 7. Said processor is used tocontrol positioning means 730, pattern generator 115, and optical sensor750. In addition, said processor analyzes data and creates a 3-D imageof a sample. In the preferred embodiment, said process is a personalcomputer.

There are alternative ways to design transmitted illuminator 700. Forexample, one can use a light source and less than three lenses toproject an image of a patterned article onto the focal plane of anobjective lens. One can also use a light source and more than threelenses to create multiple field-conjugate planes where he can place apatterned article and have its image projected onto the focal plane ofan objective lens. It is understood that these alternative illuminatordesigns are also within the scope of the present invention.

FIG. 8 is a diagram illustrating a 3-D optical microscope with atransmitted illuminator in accordance with a fourth embodiment of thepresent invention. The main difference between this embodiment and thethird embodiment illustrated in FIG. 7 is in illuminator design.Illuminator 800 provides illumination for the microscope and containsseveral components, one of which is an electronic pattern generator 604.Light source 801 generates illuminating light. Among possible choicesfor the light source are: a lamp, a fiber coupled light, a LED light, alaser, and etc. In the preferred embodiment, either a halogen lamp or afiber coupled light source is used. Lenses 802, 803, 805, and 711 ensurea uniform illumination on sample 720 and, when electronic patterngenerator 604 acts like a patterned article, project an image of thepatterned article to the focal plane of objective lens 710.Beam-splitter 806 is mounted at a 45° angle with respect to a horizontaldirection so that light from the light source is directed verticallyupward to objective lens 710.

Positioning means 730 is provided to change the relative positionbetween sample 720 and objective lens 710. As a result, differentfeatures on the sample can be brought into focus of objective lens 710.In the preferred embodiment, positioning means 730 is a motorized Zstage. As an option, a XY stage (not shown) can be incorporated into themicroscope of FIG. 8 to move sample 720 around in a horizontal plane.There are, of course, other ways to vary the relative position betweensample 720 and objective lens 710. For example, objective lens 710 couldbe mounted on a piezoelectric actuator. In such an arrangement, sample720 remains stationary while objective lens 710 moves up and down. It isunderstood by those skilled in the art that these variations are withinthe scope of this invention. Coupler 740 in conjunction with objectivelens 710 yields an image of sample 720 on optical sensor 750. In thepreferred embodiment, optical sensor 750 is either a CCD or a CMOScamera. Processor 160 is connected to the 3-D optical microscope of FIG.8. Said processor is used to control positioning means 730, electronicpattern generator 604, and optical sensor 750. In addition, saidprocessor analyzes data and creates a 3-D image of the sample. In thepreferred embodiment, said process is a personal computer.

FIG. 9A is a diagram illustrating a conventional optical microscope witha reflective illuminator. Illuminator 900 typically includes afield-stop 902 (F-Stop). Light source 901 is attached onto theilluminator. Objective turret 903, often with 4 to 6 mounting holes, isattached to microscope body 904. Objective lens 905 is threaded into oneof the mounting holes of objective turret 903. Sample stage 906 can movein X and Y direction with turning knob 907 and move in vertical (Z)direction with focusing knobs 908 or 909. Focusing knob 908 can initiatelarge step moves in the Z direction and therefore is often called coarsefocus knob. Focusing knob 909 performs small step moves in the Zdirection and therefore is often called fine focus knob. Sample 910 isseated on sample stage 906. Trinocular tube 911 is attached toilluminator 900. Two identical eyepieces 912 slide into two of the threeopenings on the trinocular tube. An operator can view the sample throughthe eyepieces. The third opening on the trinocular tube is reserved foradding a camera which is optional for a conventional microscope.

FIG. 9B illustrates modifications made to a conventional microscope ofFIG. 9A in order to turn it into a 3-D optical microscope in accordancewith a fifth embodiment of the present invention. F-stop 902 of FIG. 9Ais replaced by pattern generator 115 of FIG. 1A. Means for focusingadjustment 913 is implemented either on fine focus knob 909 or onobjective turret 903 of FIG. 9A. Some examples of means for focusingadjustment are electrical motor, piezoelectric actuator, and etc. In thepreferred embodiment, means for focusing adjustment 913 is a motorcoupled to fine focus knob 909 of FIG. 9A. It is understood that othermeans of focusing adjustment is also within the scope of the presentinvention. Coupler 914 is mounted on trinocular tube 911 and camera 915is attached to coupler 914. Finally, processor 160 is connected to themodified microscope of FIG. 9B. The processor is used to control meansfor focusing adjustment 913, camera 915, and pattern generator 115. Inaddition, said processor analyzes data and creates a 3-D image of thesample. In the preferred embodiment, processor 160 is a personalcomputer.

FIG. 9C illustrates modifications made to a conventional microscope ofFIG. 9A in order to turn it into a 3-D optical microscope in accordancewith a sixth embodiment of the present invention. Illuminator 916 isadded on top of the original illuminator of the conventional microscopeof FIG. 9A. This extra illuminator provides a means to projecting animage of a pattern created by pattern generator 115 onto and to removingthe image of the pattern from the focal plane of the objective lens. Theoptical design of illuminator 916 is very similar to that of illuminator100. For example, both illuminators have the same lenses, the same45-degree beam-splitter, the same pattern generator, and the sameoptical layout. Illuminator 916 has a pull lever 918 attached to the45-degree beam-splitter. In addition, light source 917, instead ofpattern generator 115, is connected to processor 160. In the preferredembodiment, pattern generator 115 is a mechanical pattern generator withits patterned article permanently placed in the optical path ofilluminator 916. The means to projecting and removing an image of thepatterned article from the focal plane of the objective lens is byturning light source 917 on and off. Among possible choices for thelight source are: a lamp, a fiber coupled light, a LED light, a laser,and etc. In the preferred embodiment, light source 917 is a LED lightsource capable of turning on and off at high speed.

Pull level 918 is used to pull the 45-degree beam-splitter out of theoptical path of illuminator 916 when the microscope is operating indark-field mode. Means for focusing adjustment 913 is implemented eitheron fine focus knob 909 or on objective turret 903 of FIG. 9A. Someexamples of means for focusing adjustment are electrical motor,piezoelectric actuator, and etc. In the preferred embodiment, means forfocusing adjustment is a motor coupled to fine focus knob 909 of FIG.9A. It is understood that other means of focusing adjustment is withinthe scope of the present invention. Coupler 914 is mounted on trinoculartube 911 of FIG. 9A and camera 915 is attached to coupler 914. Finally,processor 160 is connected to the modified microscope of FIG. 9C. Theprocessor is used to control means for focusing adjustment 913, camera915, and light source 917. In addition, said processor analyzes data andcreates a 3-D image of the sample. In the preferred embodiment,processor 160 is a personal computer. It is understood that illuminators100 and 600 can replace illuminator 916 in the current embodiment. Ifsuch replacement were to occur, means for projecting and removing animage of a patterned article would have to be implemented on patterngenerators 115 and 604 respectively.

FIG. 10A is a diagram illustrating a conventional optical microscopewith a transmitted illuminator. A substantial portion of the illuminatoris hidden inside microscope body 1000. Some visible components of theilluminator typically include a field-stop 1002 (F-Stop), lens 1003, andcondenser lens 1004. Light source 1001 is mounted to the entrance of theilluminator. Objective turret 1005, often with 4 to 6 mounting holes, isattached to microscope body 1000. Objective lens 1006 is threaded intoone of the mounting holes of objective turret 1005. Sample stage 1007can move in X and Y direction with turning knob 1008 and move invertical (Z) direction with focusing knobs 1009 or 1010. Focusing knob1009 can initiate large step moves in the Z direction and therefore isoften called coarse focus knob. Focusing knob 1010 performs small stepmoves in the Z direction and therefore is often called fine focus knob.Sample 1011 is mounted on sample stage 1007. Condenser lens 1004 andsample stage 1007 travel in the Z direction together under the commandof focusing knobs 1009 and 1010. Trinocular tube 1012 is attached tomicroscope body 1000. Two identical eyepieces 1013 slide into two of thethree openings on the trinocular tube. An operator can view the samplethrough the eyepieces. The third opening on the trinocular tube isreserved for adding a camera which is optional for a conventionalmicroscope.

FIG. 10B illustrates modifications made to a conventional microscope ofFIG. 10A in order to turn it into a 3-D optical microscope in accordancewith a seventh embodiment of the present invention. F-stop 1002 of FIG.10A is replaced by pattern generator 115 of FIG. 1A. Means for focusingadjustment 1014 is implemented either on fine focus knob 1010 or onobjective turret 1005 of FIG. 10A. Some examples of means for focusingadjustment are electrical motor, piezoelectric actuator, and etc. In thepreferred embodiment, means for focusing adjustment 1014 is a motorcoupled to fine focus knob 1010 of FIG. 10A. It is understood that othermeans of focusing adjustment is within the scope of the presentinvention. Coupler 1015 is mounted on trinocular tube 1012 of FIG. 10Aand camera 1016 is attached to coupler 1015. Finally, processor 160 isconnected to the modified microscope of FIG. 10B. The processor is usedto control means for focusing adjustment 1014, camera 1016, and patterngenerator 115. In addition, said processor analyzes data and creates a3-D image of the sample. In the preferred embodiment, processor 160 is apersonal computer.

FIG. 10C illustrates modifications made to a conventional microscope ofFIG. 10A in order to turn it into a 3-D optical microscope in accordancewith a eighth embodiment of the present invention. Illuminator 1017 isadded on top of the original illuminator of the conventional microscopeof FIG. 10A. This extra illuminator provides a means to projecting animage of a pattern created by pattern generator 115 onto and to removingthe image of the pattern from the focal plane of the objective lens. Theoptical design of illuminator 1017 is very similar to that ofilluminator 100. For example, both illuminators have the same lenses,the same 45-degree beam-splitter, the same pattern generator, and thesame optical layout. Illuminator 1017 has a pull lever 1019 attached tothe 45-degree beam-splitter. In addition, light source 1018, instead ofpattern generator 115, is connected to processor 160. In the preferredembodiment, pattern generator 115 is a mechanical pattern generator withits patterned article permanently placed in the optical path ofilluminator 1017. The means to projecting and removing an image of thepatterned article from the focal plane of the objective lens is byturning light source 1018 on and off. Among possible choices for thelight source are: a lamp, a fiber coupled light, a LED light, a laser,and etc. In the preferred embodiment, light source 1018 is a LED lightsource capable of turning on and off at high speed.

Pull lever 1019 is used to pull the 45-degree beam-splitter out of theoptical path of illuminator 1017 when the microscope is operating indark-field mode. Means for focusing adjustment 1014 is implementedeither on fine focus knob 1010 or on objective turret 1005 of FIG. 10A.Some examples of means for focusing adjustment are electrical motor,piezoelectric actuator, and etc. In the preferred embodiment, means forfocusing adjustment is a motor coupled to fine focus knob 1010 of FIG.10A. It is understood that other means of focusing adjustment is withinthe scope of the present invention. Coupler 1015 is mounted ontrinocular tube 1012 of FIG. 10A and camera 1016 is attached to coupler1015. Finally, processor 160 is connected to the modified microscope ofFIG. 10C. The processor is used to control means for focusing adjustment1014, camera 1016, and light source 1018. In addition, said processoranalyzes data and creates a 3-D image of the sample. In the preferredembodiment, processor 160 is a personal computer.

FIG. 10D illustrates modifications made to a conventional microscope ofFIG. 10A in order to turn it into a 3-D optical microscope in accordancewith a ninth embodiment of the present invention. Illuminator 916 isadded on top of the original illuminator of the conventional microscopeof FIG. 10A. This extra illuminator provides a means to projecting animage of a pattern created by pattern generator 115 onto and to removingthe image of the pattern from the focal plane of the objective lens. Inthe preferred embodiment, pattern generator 115 is a mechanical patterngenerator with its patterned article permanently placed in the opticalpath of illuminator 916. The means to projecting and removing an imageof the patterned article from the focal plane of the objective lens isby turning light source 917 on and off. Among possible choices for thelight source are: a lamp, a fiber coupled light, a LED light, a laser,and etc. In the preferred embodiment, light source 917 is a LED lightsource capable of turning on and off at high speed.

Pull lever 918 is used to pull the 45-degree beam-splitter out of theoptical path of illuminator 916 when the microscope is operating indark-field mode. Means for focusing adjustment 1014 is implementedeither on fine focus knob 1010 or on objective turret 1005 of FIG. 10A.Some examples of means for focusing adjustment are electrical motor,piezoelectric actuator, and etc. In the preferred embodiment, means forfocusing adjustment is a motor coupled to fine focus knob 1010 of FIG.10A. It is understood that other means of focusing adjustment is withinthe scope of the present invention. Coupler 1015 is mounted ontrinocular tube 1012 of FIG. 10A and camera 1016 is attached to coupler1015. Finally, processor 160 is connected to the modified microscope ofFIG. 10D. The processor is used to control means for focusing adjustment1014, camera 1016, and light source 917. In addition, said processoranalyzes data and creates a 3-D image of the sample. In the preferredembodiment, processor 160 is a personal computer. It is understood thatilluminators 100 and 600 can replace illuminator 916 in the currentembodiment. If such replacement were to occur, means for projecting andremoving an image of a patterned article would have to be implemented onpattern generators 115 and 604 respectively.

Operation principles of the 3-D microscopes in accordance with a second,third, fourth, fifth, sixth, seventh, eighth, and ninth embodiment ofthe present invention are similar to that of the first embodiment. Sincewe have described the latter in great detail, we will not repeat thesame description here. The key points are: (1) when a mechanical patterngenerator is used, creating a 3-D image using the microscopes of FIG.1A, FIG. 7, FIG. 9B, and FIG. 10B in accordance with the presentinvention involves the aforementioned two-pass image acquisition processof FIG. 5A and subsequent data analysis process of FIG. 5C. (2) When anelectronic pattern generator is used, creating a 3-D image using themicroscopes of FIG. 1A, FIG. 6, FIG. 7, FIG. 8, FIG. 9B, and FIG. 10B inaccordance with the present invention involves the aforementionedone-pass image acquisition process of FIG. 5B and subsequent dataanalysis process of FIG. 5C. (3) Although a mechanical pattern generatoris used, creating a 3-D image using the microscopes of FIG. 9C, FIG. 10Cand FIG. 10D in accordance with the present invention involves theaforementioned one-pass image acquisition process of FIG. 5B andsubsequent data analysis process of FIG. 5C. The reason is that on theseparticular microscopes, projecting and removing an image of a patternedarticle can be done electronically by turning the light source on andoff.

The modifications of the microscopes of FIGS. 9B to 9C and FIGS. 10B to10D in accordance with the present invention can be implemented easilyand economically on almost all conventional optical microscopes as longas they have a fine focus knob. This is a big advantage over prior artrelated to confocal microscopy. A confocal microscope is relativelyexpensive to build. It is not easy and in many cases impossible to turnan existing optical microscope into a confocal microscope. With thepresent invention, however, an existing optical microscope can be easilyturned into a 3-D optical microscope with just a few simplemodifications.

1. A three-dimensional (3-D) optical microscope comprising: at least oneobjective lens for forming an optical image of a sample; an opticalsensor capable of acquiring the optical image; a focusing adjustmentdevice for adjusting a focus position of the sample relative to theobjective lens; a pattern generator to project a pattern onto a focalplane of the objective lens; and a processor coupled to the opticalsensor, the focusing adjustment device, and the pattern generator, theprocessor configured to capture first and second images at multiple Zsteps, the first image with the pattern and the second image without thepattern, the processor further configured to generate a depth profileusing the first image and an image intensity using the second image. 2.The 3-D optical microscope of claim 1, wherein the focusing adjustingdevice includes a motorized Z stage.
 3. The 3-D optical microscope ofclaim 1, wherein the optical sensor includes one of a charge-coupleddevice (CCD) camera and a complementary metal-oxide semiconductor (CMOS)camera.
 4. The 3-D optical microscope of claim 1, wherein the patterngenerator is one of a mechanical pattern generator and an electronicpattern generator.
 5. The 3-D optical microscope of claim 4, wherein themechanical pattern generator includes one of a piece of film with apre-determined pattern and a piece of glass with a pre-determinedpattern.
 6. The 3-D optical microscope of claim 4, wherein theelectronic pattern generator is one of a liquid crystal patterngenerator and a digital micro-mirror device.
 7. The 3-D opticalmicroscope of claim 1, wherein the pattern generator is a mechanicalpattern generator linked to a motorized actuator.
 8. A three-dimensional(3-D) optical microscope comprising: at least one objective lens forforming an optical image of a sample; an optical sensor capable ofacquiring the optical image; a focusing adjustment device for adjustinga focus position of the sample relative to the objective lens; a patterngenerator to project a pattern onto a focal plane of the objective lens;and a processor coupled to the optical sensor, the focusing adjustmentdevice, and the pattern generator, the processor configured to capturefirst and second images at multiple Z steps, the first image with thepattern and the second image without the pattern, the processor furtherconfigured to generate a depth profile using the first image and colorvalues using the second image.
 9. The 3-D optical microscope of claim 8,wherein the focusing adjusting device includes a motorized Z stage. 10.The 3-D optical microscope of claim 8, wherein the optical sensorincludes one of a charge-coupled device (CCD) camera and a complementarymetal-oxide semiconductor (CMOS) camera.
 11. The 3-D optical microscopeof claim 8, wherein the pattern generator is one of a mechanical patterngenerator and an electronic pattern generator.
 12. The 3-D opticalmicroscope of claim 11, wherein the mechanical pattern generatorincludes one of a piece of film with a pre-determined pattern and apiece of glass with a pre-determined pattern.
 13. The 3-D opticalmicroscope of claim 11, wherein the electronic pattern generator is oneof a liquid crystal pattern generator and a digital micro-mirror device.14. The 3-D optical microscope of claim 8, wherein the pattern generatoris a mechanical pattern generator linked to a motorized actuator.
 15. Athree-dimensional (3-D) optical microscope comprising: means for formingan optical image of a sample; means for acquiring the optical image;means for adjusting a focus position of the sample relative to theobjective lens; means for projecting a pattern onto a focal plane of theobjective lens; and a processor coupled to the means for acquiring theoptical image, the means for adjusting the focus position, and the meansfor projecting the pattern, the processor configured to capture firstand second images at multiple Z steps, the first image with the patternand the second image without the pattern, the processor furtherconfigured to generate a depth profile using the first image and animage intensity using the second image.
 16. The 3-D optical microscopeof claim 15, wherein the means for adjusting the focus position includesa motorized Z stage.
 17. The 3-D optical microscope of claim 15, whereinthe means for forming an optical image includes one of a charge-coupleddevice (CCD) camera and a complementary metal-oxide semiconductor (CMOS)camera.
 18. The 3-D optical microscope of claim 15, wherein the meansfor projecting a pattern is one of a mechanical pattern generator and anelectronic pattern generator.
 19. The 3-D optical microscope of claim18, wherein the mechanical pattern generator includes one of a piece offilm with a pre-determined pattern and a piece of glass with apre-determined pattern.
 20. The 3-D optical microscope of claim 18,wherein the electronic pattern generator is one of a liquid crystalpattern generator and a digital micro-mirror device.
 21. The 3-D opticalmicroscope of claim 15, wherein the means for projecting a pattern is amechanical pattern generator linked to a motorized actuator.
 22. Athree-dimensional (3-D) optical microscope comprising: means for formingan optical image of a sample; means for acquiring the optical image;means for adjusting a focus position of the sample relative to theobjective lens; means for projecting a pattern onto a focal plane of theobjective lens; and a processor coupled to the means for acquiring theoptical image, the means for adjusting the focus position, and the meansfor projecting the pattern, the processor configured to capture firstand second images at multiple Z steps, the first image with the patternand the second image without the pattern, the processor furtherconfigured to generate a depth profile using the first image and colorvalues using the second image.
 23. The 3-D optical microscope of claim22, wherein the means for adjusting the focus position includes amotorized Z stage.
 24. The 3-D optical microscope of claim 22, whereinthe means for forming an optical image includes one of a charge-coupleddevice (CCD) camera and a complementary metal-oxide semiconductor (CMOS)camera.
 25. The 3-D optical microscope of claim 22, wherein the meansfor projecting a pattern is one of a mechanical pattern generator and anelectronic pattern generator.
 26. The 3-D optical microscope of claim25, wherein the mechanical pattern generator includes one of a piece offilm with a pre-determined pattern and a piece of glass with apre-determined pattern.
 27. The 3-D optical microscope of claim 25,wherein the electronic pattern generator is one of a liquid crystalpattern generator and a digital micro-mirror device.
 28. The 3-D opticalmicroscope of claim 22, wherein the means for projecting a pattern is amechanical pattern generator linked to a motorized actuator.
 29. Acomputer-readable medium storing computer instructions that, when run ona computer, generate a three-dimensional (3-D) image of a sample usingan optical microscope, the computer instructions performing stepscomprising: varying the relative distance between the sample and anobjective lens of the optical microscope at pre-determined steps; at oneor more of the pre-determined steps: capturing a first image of thesample with a pattern, which is projected onto a focal plane of theobjective lens; storing the first image in a first image array;capturing a second image of the sample without the pattern; and storingthe second image in a second image array; generating a depth profileusing the first image array; generating image intensity using the secondimage array; combining the depth profile and the image intensity togenerate the 3-D image of the sample.
 30. The computer-readable mediumof claim 29, wherein the pattern is projected onto the focal plane usingelectronic pattern generation.
 31. The computer-readable medium of claim29, wherein generating the depth profile includes extracting contrastinformation from the first image array using a maximum gradientapproach.
 32. A computer-readable medium storing computer instructionsthat, when run on a computer, generate a three-dimensional (3-D) imageof a sample using an optical microscope, the computer instructionsperforming steps comprising: varying the relative distance between thesample and an objective lens of the optical microscope at pre-determinedsteps; at one or more of the predetermined steps: capturing a firstimage of the sample with a pattern, which is projected onto a focalplane of the objective lens; storing the first image in a first imagearray; capturing a second image of the sample without the pattern; andstoring the second image in a second image array; generating a depthprofile using the first image array; generating color values using thesecond image array; combining the depth profile and the color values togenerate the 3-D image of the sample.
 33. The computer-readable mediumof claim 32, wherein the pattern is projected onto the focal plane usingelectronic pattern generation.
 34. The computer-readable medium of claim32, wherein generating the depth profile includes extracting contrastinformation from the first image array using a maximum gradientapproach.